Histograms – Extracurricular hours

• Histograms provide a view of the data density. Higher bars represent where the data are relatively more common.
  
• Histograms are especially convenient for describing the shape of the data distribution.
  
• The chosen bin width can alter the story the histogram is telling.

Histogram of extracurricular hours

Bin width

Which one(s) of these histograms are useful? Which reveal too much about the data? Which hide too much?

Shape of a distribution: modality

Does the histogram have a single prominent peak (unimodal), several prominent peaks (bimodal/multimodal), or no apparent peaks (uniform)?

Modality examples

In order to determine modality, step back and imagine a smooth curve over the histogram — imagine that the bars are wooden blocks and you drop a limp spaghetti over them. The shape the spaghetti would take could be viewed as a smooth curve.

Shape of a distribution: skewness

Is the histogram right skewed, left skewed, or symmetric?

Skewness examples

Histograms are said to be skewed to the side of the long tail.

Shape of a distribution: unusual observations

Are there any unusual observations or potential outliers?

Outlier examples

Extracurricular activities

How would you describe the shape of the distribution of hours per week students spend on extracurricular activities?

Histogram of extracurricular hours

Unimodal and right skewed, with a potentially unusual observation at 60 hours/week.

Commonly observed shapes of distributions

Practice

Which of these variables do you expect to be uniformly distributed?

1. weights of adult females
   
2. salaries of a random sample of people from North Carolina
   
3. house prices
   
4. birthdays of classmates (day of the month)

Application activity: Shapes of distributions

Sketch the expected distributions of the following variables:

• number of piercings
  
• scores on an exam
  
• IQ scores

Come up with a concise way (1-2 sentences) to teach someone how to determine the expected distribution of any variable.

Are you typical?

Typical distribution

http://www.youtube.com/watch?v=4B2xOvKFFz4

Variance and standard deviation

Variance

Variance is roughly the average squared deviation from the mean.

\[ s^2 = \frac{\sum_{i = 1}^n (x_i - \bar{x})^2}{n - 1} \]

Variance (cont.)

Why do we use the squared deviation in the calculation of variance?

Standard deviation

The standard deviation is the square root of the variance, and has the same units as the data.

\[ s = \sqrt{s^2} \]

Box plots, quartiles, and the median

Median

• The median is the value that splits the data in half when ordered in ascending order.

  \[0, 1, \textbf{2}, 3, 4\]

• If there are an even number of observations, then the median is the average of the two values in the middle.

  \[0, 1, \underline{2, 3}, 4, 5 \rightarrow \frac{2 + 3}{2} = \textbf{2.5}\]

• Since the median is the midpoint of the data, 50% of the values are below it. Hence, it is also the \(50^{th}\) percentile.

Q1, Q3, and IQR

• The \(25^{th}\) percentile is also called the first quartile, Q1.

• The \(50^{th}\) percentile is also called the median.

• The \(75^{th}\) percentile is also called the third quartile, Q3.

• Between Q1 and Q3 is the middle 50% of the data. The range these data span is called the interquartile range, or the IQR.

Box plot

The box in a box plot represents the middle 50% of the data, and the thick line in the box is the median.

Box plot of study hours

Anatomy of a box plot

Box plot anatomy

Whiskers and outliers

• Whiskers of a box plot can extend up to \(1.5 \times IQR\) away from the quartiles.

\[ \begin{aligned} \text{max upper whisker reach} &= Q3 + 1.5 \times IQR \\ \text{max lower whisker reach} &= Q1 - 1.5 \times IQR \end{aligned} \]

Outliers (cont.)

Question: Why is it important to look for outliers?

Robust statistics

Extreme observations

Question: How would sample statistics such as mean, median, SD, and IQR of household income be affected if the largest value was replaced with $10 million? What if the smallest value was replaced with $10 million?

Household income dot plot

Robust statistics

Household income dot plot

Robust statistics

Median and IQR are more robust to skewness and outliers than mean and SD. Therefore,

• for skewed distributions it is often more helpful to use median and IQR to describe the center and spread

Mean vs. median

• If the distribution is symmetric, center is often defined as the mean: mean ≈ median

• If the distribution is skewed or has extreme outliers, center is often defined as the median

  • Right-skewed: mean > median
    
  • Left-skewed: mean < median

Practice

Which is most likely true for the distribution of percentage of time actually spent taking notes in class versus on Facebook, Twitter, etc.?

Transforming data

Extremely skewed data

When data are extremely skewed, transforming them might make modeling easier. A common transformation is the log transformation.

Pros and cons of transformations

• Skewed data are easier to model with when they are transformed because outliers tend to become far less prominent after an appropriate transformation.

  
  

  # of games	70	50	25	⋯
  log(# of games)	4.25	3.91	3.22	⋯

  
  

• However, results of an analysis in log units of the measured variable might be difficult to interpret.

Mapping data

Intensity maps

Q: What patterns are apparent in the change in population between 2000 and 2010?

Source ::::::::::::::::::::::::::::::::